.. _fiftyone:

########
FiftyOne
########

We have collaborated with the team at
`Voxel51 <https://voxel51.com>`__ to integrate their tool,
`FiftyOne <https://fiftyone.ai>`__, into Lightning Flash.

FiftyOne is an open-source tool for building high-quality
datasets and computer vision models. The FiftyOne API and App enable you to
visualize datasets and interpret models faster and more effectively.

This integration allows you to view predictions generated by your tasks in the
:ref:`FiftyOne App <fiftyone:fiftyone-app>`, as well as easily incorporate
:ref:`FiftyOne Datasets <fiftyone:fiftyone-basics>` into your tasks. All image and video tasks
are supported!

.. raw:: html

  <div style="margin-top: 20px; margin-bottom: 20px">
    <video controls muted poster="https://pl-flash-data.s3.amazonaws.com/assets/fiftyone/fiftyone_poster.png" style="width: 100%;">
      <source src="https://pl-flash-data.s3.amazonaws.com/assets/fiftyone/fiftyone_long_sizzle.mp4" type="video/mp4">
    </video>
  </div>

************
Installation
************

In order to utilize this integration, you will need to
:ref:`install FiftyOne <fiftyone:installing-fiftyone>`:

.. code:: shell

    pip install fiftyone

*****************************
Visualizing Flash predictions
*****************************

This section shows you how to augment your existing Lightning Flash workflows
with a couple of lines of code that let you visualize predictions in FiftyOne.
You can visualize predictions for classification, object detection, and
semantic segmentation tasks. Doing so is as easy as updating your model to use
one of the following outputs:

* :class:`FiftyOneLabelsOutput(return_filepath=True)<flash.core.classification.FiftyOneLabelsOutput>`
* :class:`FiftyOneSegmentationLabelsOutput(return_filepath=True)<flash.image.segmentation.output.FiftyOneSegmentationLabelsOutput>`
* :class:`FiftyOneDetectionLabelsOutput(return_filepath=True)<flash.image.detection.output.FiftyOneDetectionLabelsOutput>`

The :func:`~flash.core.integrations.fiftyone.visualize` function then lets you visualize
your predictions in the
:ref:`FiftyOne App <fiftyone:fiftyone-app>`. This function accepts a list of
dictionaries containing :ref:`FiftyOne Label <fiftyone:using-labels>` objects
and filepaths, which is exactly the output of the FiftyOne outputs when the
``return_filepath=True`` option is specified.

.. literalinclude:: ../../../examples/image/fiftyone_img_classification.py
    :language: python
    :lines: 14-

The :func:`~flash.core.integrations.fiftyone.visualize` function can be used in
all of the following environments:

-   **Local Python shell**: The App will launch in a new tab in your default
    web browser
-   **Remote Python shell**: Pass the ``remote=True`` option and then follow
    the instructions printed to your remote shell to open the App in your
    browser on your local machine
-   **Jupyter notebook**: The App will launch in the output of your current
    cell
-   **Google Colab**: The App will launch in the output of your current cell
-   **Python script**: Pass the ``wait=True`` option to block execution of your
    script until the App is closed

See :ref:`this page <fiftyone:environments>` for more information about
using the FiftyOne App in different environments.

***********************
Using FiftyOne datasets
***********************

The above workflow is great for visualizing model predictions. However, if you
store your data in a FiftyOne Dataset initially, then you can also visualize
ground truth annotations. This allows you to perform more complex analysis with
:ref:`views <fiftyone:using-views>` into your data and
:ref:`evaluation <fiftyone:evaluating-models>` of your model results.

The
:meth:`~flash.core.data.data_module.DataModule.from_fiftyone`
method allows you to load your FiftyOne datasets directly into a
:class:`~flash.core.data.data_module.DataModule` to be used for training,
testing, or inference.

.. literalinclude:: ../../../examples/image/fiftyone_img_classification_datasets.py
    :language: python
    :lines: 14-

**********************
Visualizing embeddings
**********************

FiftyOne provides the methods for
:ref:`dimensionality reduction<fiftyone:brain-embeddings-visualization>` and
:ref:`interactive plotting<fiftyone:embeddings-plots>`. When combined with
:ref:`embedding tasks <image_embedder>` in Flash, you can accomplish
powerful workflows like clustering, similarity search, pre-annotation, and more
in only a few lines of code.

.. literalinclude:: ../../../examples/image/fiftyone_img_embedding.py
    :language: python
    :lines: 14-

.. image:: https://pl-flash-data.s3.amazonaws.com/assets/fiftyone/embeddings.png
   :alt: embeddings_example
   :align: center
